KAFKA-19817 Move DynamicTopicClusterQuotaPublisher to metadata module #20783
+156
−114
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JimmyWang6
force-pushed
the
KAFKA-19817
branch
from
fd909db to
bdfe7b3
Compare
@JimmyWang6 please fix the build error |
|
@chia7712 Thanks for the comment, just fixed the build error, please take another look when you're free :) |
chia7712
reviewed
Oct 30, 2025
...ata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java
Outdated
Show resolved
Hide resolved
...ata/src/main/java/org/apache/kafka/metadata/publisher/DynamicTopicClusterQuotaPublisher.java
Outdated
Show resolved
Hide resolved
server-common/src/main/java/org/apache/kafka/server/network/Session.java
Show resolved
Hide resolved
|
@chia7712 Thanks for your comments. All of them have been addressed, and please take another look. |
chia7712
reviewed
Oct 31, 2025
| quotaManagersProvider.clientQuotaCallbackPlugin().ifPresent(plugin -> { | ||
| if (delta.topicsDelta() != null || delta.clusterDelta() != null) { | ||
| Cluster cluster = MetadataCache.toCluster(clusterId, newImage); | ||
| if (plugin.get().updateClusterMetadata(cluster)) { |
There was a problem hiding this comment.
The necessity of this method is being discussed in KIP-1200. While I hope the vote succeeds, there is no harm in migrating the code though
chia7712
reviewed
Oct 31, 2025
| int nodeId, | ||
| FaultHandler faultHandler, | ||
| String nodeType, | ||
| QuotaManagersProvider quotaManagers |
There was a problem hiding this comment.
Could you consider de-coupling those components by using lambda function? for example:
public DynamicTopicClusterQuotaPublisher(
String clusterId,
int nodeId,
FaultHandler faultHandler,
String nodeType,
Plugin<ClientQuotaCallback> clientQuotaCallbackPlugin,
Runnable updateQuotaMetricConfigs
) {
this.clusterId = clusterId;
this.nodeId = nodeId;
this.faultHandler = faultHandler;
this.nodeType = nodeType;
this.clientQuotaCallbackPlugin = clientQuotaCallbackPlugin;
this.updateQuotaMetricConfigs = updateQuotaMetricConfigs;
}
@Override
public String name() {
return "DynamicTopicClusterQuotaPublisher " + nodeType + " id=" + nodeId;
}
@Override
public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
try {
if (delta.topicsDelta() != null || delta.clusterDelta() != null) {
Cluster cluster = MetadataCache.toCluster(clusterId, newImage);
if (clientQuotaCallbackPlugin.get().updateClusterMetadata(cluster)) {
updateQuotaMetricConfigs.run();
}
}
} catch (Exception e) {
String deltaName = "MetadataDelta up to " + newImage.highestOffsetAndEpoch().offset();
faultHandler.handleFault("Uncaught exception while publishing dynamic topic or cluster changes from " + deltaName, e);
}
}- we could create
DynamicTopicClusterQuotaPublisheronly if theclientQuotaCallbackPluginis existent - we wrap all callbacks in a single runnable object to avoid move many classes into
server-commonmodule
@JimmyWang6 WDYT?
There was a problem hiding this comment.
Hi @chia7712 ,
I just tried and found that clientQuotaCallbackPlugin also belongs to class ClientQuotaManager, so if we are going to de-coupe those components, the code will be shown like this:
public class DynamicTopicClusterQuotaPublisher implements MetadataPublisher {
private final FaultHandler faultHandler;
private final String nodeType;
private final Runnable updateQuotaMetricConfigs;
public DynamicTopicClusterQuotaPublisher(
FaultHandler faultHandler,
String nodeType,
Runnable updateQuotaMetricConfigs
) {
this.faultHandler = faultHandler;
this.nodeType = nodeType;
this.updateQuotaMetricConfigs = updateQuotaMetricConfigs;
}
@Override
public String name() {
return "DynamicTopicClusterQuotaPublisher " + nodeType;
}
@Override
public void onMetadataUpdate(MetadataDelta delta, MetadataImage newImage, LoaderManifest manifest) {
try {
if (delta.topicsDelta() != null || delta.clusterDelta() != null) {
updateQuotaMetricConfigs.run();
}
} catch (Exception e) {
String deltaName = "MetadataDelta up to " + newImage.highestOffsetAndEpoch().offset();
faultHandler.handleFault("Uncaught exception while publishing dynamic topic or cluster changes from " + deltaName, e);
}
}
}And the logic of the original DynamicTopicClusterQuotaPublisher will still remain in the new class updateQuotaMetricConfigs somewhere within the core module. However, such a modification might deviate from the original intent of this issue. What's your perspective on this?
There was a problem hiding this comment.
yes but DynamicTopicClusterQuotaPublisher can generate the Cluster and check the result of updateClusterMetadata
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
see: KAFKA-19817
This PR intended to move
DynamicTopicClusterQuotaPublisherto the metadata module .To prevent circular dependencies, several associated classes have been migrated as well，including
ClientSensors,ClientQuotaMetadataManager,ThrottleCallbackandThrottledChannel.Additionally, a new interface
QuotaManagersProviderhas been introduced to avoid the need to move classQuotaManagersinQuotaFactory, as doing so would have necessitated significantly more extensive changes.